Metadata-augmented multilevel Bayesian network framework for image sensor fusion and main subject detection

Automatic main subject detection refers to the problem of determining salient or interesting objects in a photograph. We have used a multilevel Bayesian network-based approach for solving this problem in the unconstrained domain of consumer photographs. In our previous work, we described building an evidential reasoning and image sensor fusion framework that uses a number of low-level and high-level image sensors to determine the intended objects-of-interest in any target image. In this paper, we will describe recent work in adding metadata-augmented reasoning processes to this framework. Many image capture devices, e.g., digital cameras, record scene metadata along with the image. This metadata can contain useful information such as whether the flash was used, orientation of the image, focal range, etc. In addition, other metadata such as indoor-outdoor, orientation, and urban-rural classification can be generated using image understanding algorithms, or user annotation. We present a Bayesian network-based approach that accurately models the system and allows for metadataaugmented sensor integration in an evidential framework. The system seamlessly operates in the absence or presence of metadata with no user intervention required. We present subjective and analytical results that show the performance improvements achieved when scene metadata-augmented reasoning processes are used.